Compiling MPI for Many-Core Systems

Processors with multiple (or many) cores and shared memory are becoming ubiquitous across the computing spectrum. MPI, the current de facto programming model for scalable parallel applications, enforces copies between source and target processes and thus can not fully utilize shared memory and cache architectures of modern machines. To enable MPI-based programs to more fully exploit features of multiand many-core architectures, we present a compiler-based transformation that transforms MPI processes into threads and fuses message serialization and deserialization loops such that send and receive calls can be replaced by direct memory accesses. Our compiler replaces most of the MPI communication functions with direct load/store accesses and our runtime provides a threaded MPI implementation to handle the remaining functions. We show the utility of our transformation with two applications, a molecular dynamics code, MiniMD, and a two-dimensional parallel fast Fourier transform (FFT). Our benchmarks show that our loop fusion techniques reduce communication times up to 43% for MiniMD and up to 59% for the FFT on modern multi-core systems. Our techniques will enable the automatic transformation of existing MPI codes to take advantage of modern shared memory architectures. In the future, this approach will aid in the transformation of MPI codes to a hybrid communication model that achieves high performance on a wide range of systems ranging from individual nodes to very large clusters of many-core nodes.